The invention relates to neural network architecture, and more precisely, to the structure and operation of weighting interconnections in a frequency-based network of neural processors ("neurons").
Neural hardware structures currently fall into three main categories: analog, digital, and optical. Analog structures use voltage/current as the weighted variable between neurons and multiple interconnects for routing potential from the neurons of one layer to the neurons of the next layer in the system. Digital architectures use counting structures as the weight variable between neurons and multiple interconnection busses for routing values between neuron layers in the system. Optical structures use holographic refraction as the weight variable between neurons and multiple light interconnects for routing values between neuron layers.
A new frequency-based neural architecture independent of these technologies has been initially disclosed and described in U.S. patent application Ser. No. 611,213, filed Nov. 8, 1990. That application describes a neural processor element or neuron whose activity state corresponds to the frequency of a pulse stream emitted by the neuron. The neuron changes its state in response to excitory (E) signals and inhibitory (I) signals. These signals are pulse streams of opposite polarity, with an E pulse stream driving forward the action of the neuron, while an I pulse stream reverses or inhibits the forward action of the neuron.
The new frequency-based neural architecture is intended to be implemented in a silicon-based, integrated circuit technology such as CMOS. A significant challenge presented by this new architecture is to optimize the interconnect structures which conduct signals representing the synaptic weights of one level of an M.times.N neural network matrix to another level. In prior art neural technologies, the interconnect structures account for the majority of device area consumption and contribute significantly to process bottlenecking. A need is manifest for an interconnect architecture flexible and modular enough to accommodate evolving neural network system architectures and to allow scaling up to large-sized systems through assembly and interconnection of smaller sub-systems.